Non-linear Matrix Equations: Equilibrium Analysis of Markov Chains

1. Introduction: In the research area of one dimensional stochastic processes, Markov chains acquire special importance due to large number of applications. One of the simplest possible continuous time Markov chains, namely birth-and-death process arises naturally in many queueing models. Such a Markov chain has an efficient recursive solution for the equilibrium probabilities. Specifically the equilibrium probabilities form a geometric sequence. The common ration/recursion constant is the solution of a quadratic equation. Evans and Wallace considered a stochastic process called the Quasi-Birth-and-Death (QBD) process as a natural generalization of the birth-and-death process. They showed that for a block-Jacobi generator of continuous parameter Markov processes (called QBD processes), the stationary probability vector X may be partitioned into 1 x n vectors k x , 0  k which are given by 0   k for R x x k o k ……………..(1.1) where the square matrix R is the minimal non-negative solution of a matrix-quadratic equation. An equilibrium probability vector X , which satisfies equation (1.1) will be called a matrix-geometric probability vector. A natural question which arises is whether the matrix geometric recursive solution exists for the equilibrium probabilities of a more general class of Markov processes. Marcel Neuts has shown [Neu1] that a matrix geometric recursive solution exists for the equilibrium probabilities of a large class of processes, called G/M/1-type Markov processes. This is fortunate since these processes provide good stochastic models for various problems arising in queueing and inventory theories. The state space, E of a G/M/1-type Markov process has the following form: } 1 , 0 :) , ({ n j i j i E     ………………………(1.2) in which " n " is finite but otherwise arbitrary.This state space can be clearly decomposed into levels by performing a lexicographic partitioning on the first state variable. For each level k, an equilibrium probability vector k